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ABSTRACT • ^ 

The derivation and analysis presented in thi^ report 
allow the ejcperim^nter ' to perform multlf actor contrasts on ^ 
correlation coefficients for sev.eral populations in an experimental 
desi<)n*' Cells of the design'^iconslTitut e independent groups of subjects 
on which two measures hav^ been taken and correlated* Measures need 
not be identical for each group* An example is giVen in irhich the 
dependent variable represents the correlation between --attitude and 
achievement for si^^econdary school science classes stratified by 
ability (three levels)* and grade level (two levels) • An overall' 
relationship was observed between science attitude' and knowledge of 
science for junior and senior high school students of sciencej 
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Multifactor Analysis of Differences 
Between Correlation Coefficients 



The general procedure for analysis of differences between 
correlation coefficients with blo.cking factors present is oTrfelined. * 
Ceais of the design constitute independent groups of subjectsTon 
which two measures have been takerf and correlated. Measures rieed 
not be identical for each group. An example is given-in which the 
dependent variable represents the correlatiCn between attitude and 
achievement for six secondary school groups,' stratified by ability 
(three levels) and grade level (two levels). * ' Vj 

<• ' * 

A researcHer may be interested in 'coi|paring^ correlation coefficients 

among several populations which have been sampled using a factorial design.. 

Coefficients will normally be calculated for the same Variables measured 

within each, population. , The researcher may be in'ferested in the effects » 

due to inclusion in specific levels of the design factors. Alternatively, 

the correlations calculated may be based upon different meesu^s for each 

population, and the researcher may be interested in trfends or differences 

between l^vels^ / ^ , ^ 



- Pr&)fi6 us Research ' ' . ' ^ " 

Since the ^theory of quadratiV^brms is necessary to the derivations 
in the paper,^ ^ome background is^tieeded. Box (i953) discussed tha quadratic ' ' 
form # . / / 

9 = z'Mz ^ /-X^ (1) ' . - 

Where is a vector of normally distributed random variables with mean, zero 

and variance-covariance, diagonal matrix M is a positive semi^def ijiite 'y^ . 

matrix. Box showed *Q is distributed as a linear* * sum of chi-s^uare random ' , , 

• . . . J • 

variables 



where 'X' IS the i-th latent root of' the matrix 



/ 



,(3) 



I 



for the <il\kracteristic equation. - ' ' , 

det |eW - XI I = 0. ^ , " ; ,(4) '• 

The st^atistic 

' I ' ' ' * 

I , Q = E M (2 - Z.)^ ' * (5) ^ . 

• ■ • • i=l • ' ► ■ . 

is a miadratic form in which the latent roots are \^ = M^a^ ^ for diagonal 
M and E matV^ces. Box^showed that the distribution of Q is given approxi- 
mately by v 

, Q gX (h) where , ' \" (6) 



I 

Z X 2 



\ 



8 r -V- '^^^ . "'^ - (7) 



. n 
' i=i ^ 



I 

i=l 

b = - . • • (8) 



\ 



"I 

^^i 



.1X2 



i=l 



Among specific references, to corre],atior;^OTnparisons, Hays (1963) give 
the statistical procedure for, testing ?he equivalence of *J independent 
correlation coefficients; Marascuilo (1966) discussed tjie x'^ analog of 
Scheffe's theorem for multiple cotnpa'rfsons among correlation coefficients 
of K independent bivariate normal populations. The statietit he used ±s 



■ ' ' ^o-=j/v3) i\- \ y . .(9) 

k=l ' k o ^ • ^ . ^ 

• ■ ■ X ■ ■ " ' ' 

' ^ l+r 

2^= 1/2- Log k - , (loV 

■ k • ^ ..1-r, 



K 



. E (n. -3). ir ' ■ ■ 

Z ^ = k=l • ^ '^k : ■ ; ..(11) • 



\ where K ' ' ^ . ' 

^ * \ sample size for k-th sample, ' 

Z = Fisher - Z traitisf ormatibn of the correlations'coef f icient 
. for the k-th sampl^and ^ • ^ ' 

Z = common estimate *of tl^ k transformed correlations.. 

\ * ^ The U-statdstichas an approximate distribution with k-1 degrees 
of fre-edom. ^derivation of this follows direqtly from Box's theorem on 
the approximate distribution of the quadratic iorm for the contrasts of (9) 

^ *fhomag^and Stanley (]^69) were interested in sex and race differences 
ir\ correlations of variables predicting college freshman grade point 
average. They transformed the coefficients to Ff^her Z-statistics and 
compared meanr; differences within sex anS^r^ace* No formal model was * ' 
developed, however. A^^formal statement of such a model seems in order. 

, • * ' • A Stati stical Model 

."Subjects ^re randomly assigned 'to* levels of a m(iJ.tifactoi? experiment. 
Two measures ar^-Hiadq on each* subject and the Measures correlated. The 
experimenter is interested in differences* in the correlations ^due to the 



S/ one 



factors and in possiblerinte^^acti-on. Each cell ih .the^design- contains^ 
'^observation/' the correlati^^ coefficient. Since no within-ceU variance 
exists for\a single observation irf such ^layout, no indep,endent ei;ror term, 
can be specified ordinarily. Since variance error^ are known, howe^Xer, ' 
they will be utilized. ' . " ^ ' * * 

For a two-way layout, to be used a& the basic example for, the r'emaind^ 
ot the paper, the Fisher-transformed correlation is decomposed into components 

where the a, 6 and dB'^terras aTe, factorial effects in the transformed oorreja- 

tions. Alsd, the staAdard error is- a„ = 1 / / n^^-3' for the ii-th cell- 

' • / ^' ^ii f 

observation. ^ ' ^ \ * / ^ ' 

Under .t^e usual constraints on^ the treatihent parameters, the mpdel 

• ^ ' \ ^ • 

represented -by (12) is similar to ^he fixed effects ANOVA mod^l/ NorraaUty 

holds, except in extreme situations l^ee Norris" and Hjelm., 1960). Errors 

are as^med to be independent and within-cell variances are homogeneous if^"" 

and only. if 'n^. = n for all i and ' ' * * • 

. * • \ * ' 

One seeming ^ advantage of working with a meti^ic-free statistic ^uch as 

the correlation coefficioot^ is tftat Coefficients can be compared directly, 

even tfiough the original measured used to calculate the coefficients were 

different. This is the informal proc^ure useS J^n construct validation* 

For eitample, correlation-^ 'between attitude and achievement would nec^sarily 

be based on different instruments in comparing eight-year-old children and 

adults. Putting aside questions of unreliability of measurement, differences- 

iV^rrelation might provide results of theoretical interest, while clirect 

comparisons of the original variabl-es ar^ difficult. 



f ' ' * * . Equal Cell Sizes ' 1 ' . 

\ When sample -sizes used to compute .the (f-ell correlations are eqqal for 
all cdlls (n^j=n), mean squares for effects may be, calculated.. in the usual 
manner. It ,is then easy to show thatj^a2) ^educes under null hypothesi 



to 



. _* ^^effect .. ' 2 
^effect. - . 1 , Ht K> ' (13)' 

' ^^rj^ ' etteclr 



In the two-way layout , . ' T 

°A 'n- ■ . - (14) : ■ : 



, MS 2 " 

^ These statistics may ^ shown to be Marascuilo's U-statistics, as well. 
The tests are independent, since the .x^- statistics are independent. An- 
additional ' test of interest ma^^e^m^xkr" on the grand mean Z , 

^ Z . Z * * ^ ' ■ 

r. . p * ' , ^ 

^ = 1^ ^ • . , • (17) , 

/ ^IJ(n-3) ^ 



which is a normally distributed statistic. Thus, one 'tan test the 

\ \. ^ . • ' * * 1 

. overair coi;relation for several -populations. 
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Unequal Cell Slzeg (Or Proportional Cell Sizes) • 

Analysis, of the two-way crossed design Is, based on the known variance , 
Bath Fisher-rtransformed correlation./ Box (1953) o^tlined the analysis for 
(the nonorthogonal one-way sign using tfle ' F-approxiin.ation fojr the rati© 
of tiwo quadratic forms^, S-inpe^the variances ar^ not estimated but* known 
for the transfotmed correlations (and lirfear combinatioij^ ,of them), \ . " ' 
'the Box theoretn given In (6) may be applied,') Searle (1971) and Marks (1974), 
have discussed the major ways of approaching the nonorthogonal^ two-factor 
analysis, of Variance.; The main, ef fe^cts mo^e;r sefems piost amenable to so-Jution - 
and intet-pretation here. . ♦ * ^* * • 

A , . . ' ■ , A 

The quadratic »foi;ms f or ,the treatment affects in the two-way layout with 
'* <^ % 

unequal sample -size In the cells under the null hypotiT^sfs 



- z 5 

r 



r 



V 



where Z = E (n..-^3) Zv.. /N', 
i. j=l ^' ^ 



(21) 
(22) 

(23)"* 



- Z = ^-"(n.. -3) Zr /N, 
' j 



Zr.. = ZE (ri., -3) /N 



(24). 



C25) 



( 



N = E (n; -3) 
j=l. 



(26) 



N = EE (n, .-3) 



and 



' . (27) 



(2S) 
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Following Sheffe' (1959) ' 



From Box (1954) The i/attix W is 'i ,/ 



ij ^i,V- ^i.^\ * 



when 2 ' 




(29) 



(30) 



(31) 



The^ latent /rcr&t^* for W are then calculated from (30) and^he^ approximation given 
in (9)* is/caiculated, uSing- one degree of freedom for each contrast. 

A simiMr statistic for^the effect Q- m^y be calculated. Since h will ' 

generari^JL^ not be integer-Valued, ^ the simplest procedure will be to interpolate 

betweefh the nearest integers to find critical values, 

7 ' . ' * 

The test of hypothesis is over ^the whole set of contrasts 'about the grand 

• ' ' • '2 ^ • . . . 

meaa. Confidence" intervals and contrasts baswd on ^the x analog to Scheff^'s. 

/ ' * ' ^ * 

Theorem may be; calculated, as noted by Marasciulo (1966). ^ 
The test fo^ overall correlation is given by ' 

(32)^ 



Z -Z 
r . . '0 



where 



. 2 



d 2.. 



1 Ij 



i=i j=l ""ij"^ 



1_ / T-J 
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. ' - \ ' ' 8 ■ 

» . ' « * 

^ and 2 is a normally-distributed variable with mean zero and variance unity, *^ 

Since variance^ are unequal in the proportional design,, the approach ^ 

outlined here seems ,to be appropriefte* Al^tefnatively , one mlg^t -use a 

procedu,re such as unweighted -means analysis, if proportions are close to one, . 

^ ' . ) , .\ ' 

K ' . ' Illustrative Example \ ' ' ^ . 

As part of a National Science Foundation grant, Welch and Gullickson 

(1973) described a testing program in'wlTicii randomly selected junior an^ ' 

senior high school classes *in fifteen states were 'given a series of attitude 

and acl^ievement measures in science 'and mathematics,' Each science class was 

randomly* divided into thirds* eacji third taking ^he^ Learning^ Attitude InvBn- 

*tory or LEI (Anderson, ,1971>/ the Science Process Inventory oi^ SPI (Welch , 

and Fellah 1967), ' and the Test of Achievement in Science or TAS (L^wrenz, 

1971), Two^ forms of the TASl were developed — Form I for the eighth ^grade 

level and Form'll for the eleventh grade l^vel/ Items were. drawn from the 

released it^ms of the National Assessment of Educational Progress (1970^. 

KuJer-Richard^on reli-'abilities reported are .87 for both ^orms-'^^nd II , 

i . 

(Garibaldi, 1974),* Since students generally did not. take all tests,- class v. 
'means were -osed as the units of analysis, * . 

f • Classes were divided into low, middle, and hi^ thirds on the fiasis of 

\ 

a:AS achievement scores, 'division ^performed separately for each Form./ This 

produced a 2x3 -^ctorial design with grade level and ability grou^ as factors. 

1 

CQrrelation coefficfients were computed within-cell- between the TAS and- SAX - 
^ with class meajis as unit of analysis. ^ Thl^ correlation gives a measure of 
association between the classes attitude toward science and knowledge of 
^ science.' Coefficient values and sample sizes are gi^ven in Tablel^ 

The analysis procedure for equal cell sizes was performed on the corre- 
. lation coefficients; 'The Ut-statistics for effects are given in Table 2* » ' ' ^ 

ERJC • 11 ' . ■ . • 



TABLE 1 

Correlations Between SAI afid TAS 
for Two Factors, Achievement and' Grade Level 



Grade level - 



Achievement Level 



Junior High 
(^bh grade) 



Low. 
r = .006 
(n = 35) 



Z = .006l 

- r 



Middle 
.247 
(35) ^ 
."252 



High 
.218 
(35) 
.222 



■Marginals 

y. = .160 



Senior High 
(11th grade) 



^ tlarginals 



.4jD1 
(35) ■ 



= .425 



Z . = ,216 
/.I 



,.418 
(35) 
.445 



Z = .349 
^2. 



.3^Q 
(35) 

.412 ; 



-.3 



7. = .427 



Zj. = .294 



Z = Fisher-transt^rmed value 



sample size in parentheses 
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. TABLE "2 - 
Statistics for Effects oil Correlation 
Between AchieVqpierit ^and Attitude 




Grand Mekn'. 
Achi^emeat 
Grade 

Achievement 
Grade 

Total ' ^ 



Statistic • 

Distribution "d.f. S.S. 



Nprmal 
2 

- X 



1 

2 
1 

2 

- 6 



.019 
.107 

.0178* 
.1438 



Statistic 
Value 



^56.45 
.304 
3.45 

.570 



Probability' Under 
Null Hypothesis 

< .01 

>'.80- 

< .07 > 

> .7 
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The grand mean of Fisher-transformed coefficients was signif-icantly greater 

V 

than zero (p < •01), indicating an overall relationship * between science ' 

attife«tte ^and knowledge"^of science for junior ^ahd senior high school students 

, of science. Na difference due to acliievement level was found, but the 

grade difference was significant (p < .07). Senior high school correlations 

w^e significantly higher than junior high correlat.ions. ,|^o achievement by 

gVade interaction was found. 

The higher correlations for senior high level students is probably due 

to the selection which takes place by eleventh grade. Only those students 

with some interest in science elect biology, physics, or chemistry, those 

subjects tfie^hlgli school sample had elected. 

* Cone lusion * - 

The derivation and anaXysi'S presented here allow the experimenter ta 

'perform multifactor contrasts on correlation qoefffcients for several 

t 

populations in an experimental design. The* utility "of the technique is 
dempnst^ted for jcross-grade level comparisons where the individual measures 
sMch as achievement tests from which the congelations are computed, are 
non-comparable by themselves. . ' 
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